Can the Implicit Association Test Measure Automatic Judgment? The Validation Continues.

In this commentary, we welcome Schimmack's reanalysis of Bar-Anan and Vianello's multitrait multimethod (MTMM) data set, and we highlight some limitations of both the original and the secondary analyses. We note that when testing the fit of a confirmatory model to a data set, theoretical justifications for the choices of the measures to include in the model and how to construct the model improve the informational value of the results. We show that making different, theory-driven specification choices leads to different results and conclusions than those reported by Schimmack (this issue, p. 396). Therefore, Schimmack's reanalyses of our data are insufficient to cast doubt on the Implicit Association Test (IAT) as a measure of automatic judgment. We note other reasons why the validation of the IAT is still incomplete but conclude that, currently, the IAT is the best available candidate for measuring automatic judgment at the person level.

Does discrimination beat association in the IAT? The discrimination-association model reconceived.

The discrimination-association model (DAM; Stefanutti et al. 2013) disentangles two components underlying the responses to the implicit association test (IAT), which pertain to stimuli discrimination (the strength of the association of the stimuli with their own category) and automatic association (the strength of the association between targets and attributes). The assumption of the DAM that these two components sum into a single process generates critical drawbacks. The present work provides a new formulation of the model, called DAM-4C, in which stimuli discrimination and automatic association are separate, independent, and competing processes. Results of theoretical and simulation studies suggest that the DAM-4C outperforms the DAM. The IAT effect is found to vary with the association rates of the DAM-4C and not with those of the DAM. The parameters of the DAM-4C fitted on data from a Coca-Pepsi IAT are found to account for variance in brand attractiveness, taste preference, and cola choice that is not accounted for by the D score and the diffusion model. In addition, the association rates estimated on data from a Black-White IAT are in line with expectations.

Data from a three-wave complete longitudinal design survey on career calling and related constructs (N = 6368).

This dataset provides de-identified raw responses to a non-anonymous three-wave online survey with a 12-month time lag. Data collection was part of a larger project on the development of career calling in Italian college students. The first wave was collected during the fall of 2014. Participants were bachelor's or master's students enrolled in 24 different study domains and 4 different universities. Sample sizes for each wave are NT 1 = 5,886, NT 2 = 1,700 and NT 3 = 881, 434 participants provided valid responses at all the three waves. Consent form was electronic. Dataset and codebook can be found here: https://osf.io/v56du/. The sample is mainly composed of women (63.8%, at Time 1). Participants' mean age at Time 1 was 23.37 years (SD = 5.39). The survey was in Italian and included multiple-item measures of career calling, intrinsic and extrinsic motivation, social support, engaged learning, clarity of professional identity, and quality of mentorship. Socio-demographic information and academic performance indicators are provided. The dataset is necessary to reproduce previously published results (Vianello et al., 2018) and can be useful to 1) investigate cross-cultural differences between college students from Italy and other countries; 2) investigate person-level variability in predictors and consequences of change in the variables collected over time; 3) develop and/or validate new statistical models for longitudinal data; and 4) develop and/or test original theoretical hypotheses.

Validity and measurement invariance of the Unified Multidimensional Calling Scale (UMCS): A three-wave survey study.

The accumulation of scientific knowledge on calling is limited by the absence of a common theoretical and measurement framework. Many different models of calling have been proposed, and we do not know how much research results that refer to a specific model are generalizable to different theoretical accounts of calling. In this article, we investigate whether two leading models of calling tackle the same construct. The two models were merged into a comprehensive framework that measures calling across seven facets: Passion, Purposefulness, Sacrifice, Pervasiveness, Prosocial Orientation, Transcendent Summons, and Identity. We then developed the Unified Multidimensional Calling Scale (UMCS) drawing from previous published items. Across two surveys involving college students (N = 5886) and adult employees (N = 205) the UMCS was proved to be valid and reliable. We also observed that the UMCS is invariant across time and calling domains. Finally, we found that facets of calling have very different relationships with outcomes and concurrent measures, suggesting that results obtained with a smaller set of facets are not generalizable to the higher-order construct of calling or to a different model that does not share the same facets.

A multi-method multi-trait test of the dual-attitude perspective.

The dual-attitude perspective posits that it is useful for research and theory to assume two distinct constructs: explicit and implicit attitudes (or automatic and deliberate evaluation). Much evidence supports this perspective, but some important tests are missing, casting doubts on studies that relied on the perspective for inference. We used a multimethod multitrait design to extensively test the validity of the dual perspective. The dataset (N = 24,015) included measurements of attitudes in 3 domains (race, politics, the self) with 7 indirect measures, and at least 3 self-report measures for each attitude domain. The dual-attitude model fit the data better than a single-attitude model. Six of the 7 indirect measures were related to the implicit construct more than to the explicit construct. The evidence supports the dual-attitude perspective, bolsters the validation of 6 indirect measures, and clears doubts from countless previous studies that used only one indirect measure to draw conclusions about implicit attitudes. (PsycINFO Database Record

Data from a pre-publication independent replication initiative examining ten moral judgement effects.

We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory's research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires. Results revealed a mix of reliable, unreliable, and culturally moderated findings. Unlike any previous replication project, this dataset includes the data from not only the replications but also from the original studies, creating a unique corpus that researchers can use to better understand reproducibility and irreproducibility in science.

Response to Comment on "Estimating the reproducibility of psychological science".

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.

GRace: a MATLAB-Based Application for fitting the discrimination-association model

The Implicit Association Test (IAT) is a computerized two-choice discrimination task in which stimuli have to be categorized as belonging to target categories or attribute categories by pressing, as quickly and accurately as possible, one of two response keys. The discrimination association model has been recently proposed for the analysis of reaction time and accuracy of an individual respondent to the IAT. The model disentangles the influences of three qualitatively different components on the responses to the IAT: stimuli discrimination, automatic association, and termination criterion. The article presents General Race (GRace), a MATLAB-based application for fitting the discrimination association model to IAT data. GRace has been developed for Windows as a standalone application. It is user-friendly and d es not require any programming experience. The use of GRace is illustrated on the data of a Coca Cola-Pepsi Cola IAT, and the results of the analysis are interpreted and discussed (AU)

No disponible

GRace: a MATLAB-based application for fitting the discrimination-association model.

The Implicit Association Test (IAT) is a computerized two-choice discrimination task in which stimuli have to be categorized as belonging to target categories or attribute categories by pressing, as quickly and accurately as possible, one of two response keys. The discrimination association model has been recently proposed for the analysis of reaction time and accuracy of an individual respondent to the IAT. The model disentangles the influences of three qualitatively different components on the responses to the IAT: stimuli discrimination, automatic association, and termination criterion. The article presents General Race (GRace), a MATLAB-based application for fitting the discrimination association model to IAT data. GRace has been developed for Windows as a standalone application. It is user-friendly and does not require any programming experience. The use of GRace is illustrated on the data of a Coca Cola-Pepsi Cola IAT, and the results of the analysis are interpreted and discussed.

Overweight people have low levels of implicit weight bias, but overweight nations have high levels of implicit weight bias.

Although a greater degree of personal obesity is associated with weaker negativity toward overweight people on both explicit (i.e., self-report) and implicit (i.e., indirect behavioral) measures, overweight people still prefer thin people on average. We investigated whether the national and cultural context - particularly the national prevalence of obesity - predicts attitudes toward overweight people independent of personal identity and weight status. Data were collected from a total sample of 338,121 citizens from 71 nations in 22 different languages on the Project Implicit website (https://implicit.harvard.edu/) between May 2006 and October 2010. We investigated the relationship of the explicit and implicit weight bias with the obesity both at the individual (i.e., across individuals) and national (i.e., across nations) level. Explicit weight bias was assessed with self-reported preference between overweight and thin people; implicit weight bias was measured with the Implicit Association Test (IAT). The national estimates of explicit and implicit weight bias were obtained by averaging the individual scores for each nation. Obesity at the individual level was defined as Body Mass Index (BMI) scores, whereas obesity at the national level was defined as three national weight indicators (national BMI, national percentage of overweight and underweight people) obtained from publicly available databases. Across individuals, greater degree of obesity was associated with weaker implicit negativity toward overweight people compared to thin people. Across nations, in contrast, a greater degree of national obesity was associated with stronger implicit negativity toward overweight people compared to thin people. This result indicates a different relationship between obesity and implicit weight bias at the individual and national levels.

Implicit sexual attitude of heterosexual, gay and bisexual individuals: disentangling the contribution of specific associations to the overall measure.

The article aims to measure implicit sexual attitude in heterosexual, gay and bisexual individuals. A Many-Facet Rasch Measurement analysis was used to disentangle the contribution of specific associations to the overall IAT measure. A preference for heterosexuals relative to homosexuals is observed in heterosexual respondents, driven most by associating positive attributes with heterosexuals rather than negative attributes with homosexuals. Differently, neither the negative nor the positive evaluation of any of the target groups play a prominent role in driving the preference for homosexuals observed in gay respondents. A preference for heterosexuals relative to homosexuals is observed in bisexual respondents, that results most from ascribing negative attributes to homosexuals rather than positive attributes to heterosexuals. The results are consistent with the expression of the need for achieving a positive self-image and with the influence of shared social norms concerning sexuality.

Preferring thin people does not imply derogating fat people. A Rasch analysis of the implicit weight attitude.

OBJECTIVE: The meaning of the implicit weight attitude in individuals of different weight by distinguishing the contribution of positive and negative associations to the overall measure was investigated. DESIGN AND METHODS: The implicit weight attitude was assessed using the Implicit Association Test. A Rasch model was used to identify which stimuli most affected the implicit measure. Explicit attitudes were assessed with a thin-fat preference scale, and thermometer scales for both thin and fat people. The sample consists of 510 individuals categorized according to their BMI. RESULTS AND CONCLUSION: A significant implicit preference for thin relative to fat people was observed in all weight groups. In normal weight and obese individuals, the preference was mostly affected by positive stimuli (more easily associated with thin than with fat people). In underweight individuals, the preference was mostly guided by positive (more easily associated with thin than with fat people) and negative (more easily associated with fat than with thin people) stimuli. In overweight individuals, all stimuli contributed to the preference in a similar way. In all weight groups, the implicit weight attitude correlated with the explicit preference and/or the thin thermometer, whereas it never correlated with the fat thermometer. A pro-thin bias was observed in normal weight and obese individuals, whereas both a pro-thin and an anti-fat bias were observed in underweight individuals. A clear preference for thin people relative to fat people was observed in overweight individuals. Therefore, uncritically interpreting the implicit preference for thin people as a sign of derogating fat people might be misleading.

A discrimination-association model for decomposing component processes of the implicit association test.

A formal model is proposed that decomposes the implicit association test (IAT) effect into three process components: stimuli discrimination, automatic association, and termination criterion. Both response accuracy and reaction time are considered. Four independent and parallel Poisson processes, one for each of the four label categories of the IAT, are assumed. The model parameters are the rate at which information accrues on the counter of each process and the amount of information that is needed before a response is given. The aim of this study is to present the model and an illustrative application in which the process components of a Coca-Pepsi IAT are decomposed.

Positive associations primacy in the IAT: a many-facet rasch measurement analysis.

Two studies investigated the different contribution of positive and negative associations to the size of the Implicit Association Test (IAT) effect. A Many-Facet Rasch Measurement analysis was applied for the purpose. Across different IATs (Race and Weight) and different groups of respondents (White, Normal weight, and Obese people) we observed that positive words increase the IAT effect whereas negative words tend to decrease it. Results suggest that the IAT is influenced by a positive associations primacy effect. As a consequence, we argue that researchers should be careful when interpreting IAT effects as a measure of implicit prejudice.

The many-facet Rasch model in the analysis of the go/no-go association task.

This article provides a many-facet Rasch measurement (MFRM) analysis of go/no-go association task (GNAT)-based measures of implicit attitudes toward sweet and salty food. We describe the statistical model and the strategy we adopted to score the GNAT, and we emphasize that, when analyzing implicit measures, MFRM indexes have to be interpreted in a peculiar way. In comparison with traditional scoring algorithms, an MFRM analysis of implicit measures provides some additional information and suffers from fewer limitations and assumptions. MFRM might help to overcome some limitations of current implicit measures, since it directly addresses some known issues and potential confounds, such as those related to a rational zero point, to the arbitrariness of the metric, and to participants' task-set switching ability.

National differences in gender-science stereotypes predict national sex differences in science and math achievement.

About 70% of more than half a million Implicit Association Tests completed by citizens of 34 countries revealed expected implicit stereotypes associating science with males more than with females. We discovered that nation-level implicit stereotypes predicted nation-level sex differences in 8th-grade science and mathematics achievement. Self-reported stereotypes did not provide additional predictive validity of the achievement gap. We suggest that implicit stereotypes and sex differences in science participation and performance are mutually reinforcing, contributing to the persistent gender gap in science engagement.

The sorting paired features task: a measure of association strengths.

The sorting paired features (SPF) task measures four associations in a single response block. Using four response options (e.g., good-Republicans, bad-Republicans, good-Democrats, and bad-Democrats), each trial requires participants to categorize two stimuli at once to a category pair (e.g., wonderful-Clinton to good-Democrats). Unlike other association measures, the SPF requires simultaneous categorization of both components of the association in the same trial. Providing measurement flexibility, it is sensitive to both focal, attended concepts and nonfocal, unattended stimulus features (e.g., gender of individuals in a politics SPF). Three studies measure race, gender, and political evaluations, differentiate automatic evaluations between known groups, provide evidence of convergent and discriminant validity with other attitude measures, and illustrate the SPF's unique measurement qualities.

Assessing the impact of replication on implicit association test effect by means of the extended logistic model for the assessment of change.

The present study consists of a longitudinal investigation in which implicit association measures were collected by means of the implicit association test (IAT), with the purpose of assessing the impact of replication on the test effect with the same participants. The extended logistic model for the assessment of change (ELMAC), a procedure that belongs to Rasch modeling, was applied. By means of the ELMAC application, the existence of latent traits characteristic of the impact on the IAT effect (due to replication through time) is shown; more precisely, the model time parameters show the decrement of the IAT effect through time, due to the decrease of the implicit preference in both conditions, compatible and incompatible, associated with sweet foods and salty foods. The results confirm that in the long run, the incompatible condition latency may approach the compatible condition latency, causing a significant attenuation of the IAT effect.